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PAPER IDENTIFICATION INFORMATION TO ASSOCIATE A 
PRINTED APPLICATION WITH AN ELECTRONIC APPLICATION 

FIELD OF THE INVENTION 

[0001] The present invention relates to the field of printing hardcopy 
representations of electronic applications; more particularly, the present 
invention relates to taking an electronic application program and printing 
them on a piece of paper having a preprinted pattern thereon and 
automatically updating the electronic application with edits made to the 
hardcopy representation. 

BACKGROUND OF THE INVENTION 

[0002] Recently, a company named Anoto developed a Bluetooth-equipped 
pen writing system, [insert Bluetooth cite] This writing system allows 
vendors to create pen services that are enabled by end users writing on 
specially encoded paper with the Anoto pen. The paper is encoded with a 
unique dot pattern (the full dot pattern is extremely large). By the pen 
reading these dots, through an integrated camera, the pen system can 
associate a particular function or application with this unique position 
within the pattern. The pen sends this position information to the internet 
(via a Bluetooth link to a cellphone, personal digital assistant (PDA) or 



personal computer (PC)) where a Paper Lookup Server (PLS) associates the 
particular pattern location with a service to be performed, and sends the 
information along to a server that then performs the desired services. An 
example of such a service could be a sheet of paper that has a writing field, 
an address field, and send button. A user would take this sheet of paper and 
create a memo by writing in the "writing field" (e.g. Hi Mom, will be home 
at 8), writing an address in the "address field" (e.g., Mom@home.com) and 
then checking the "send" box. Once the "send" command was executed the 
pen would then send the data to the PLS which would direct the data to an 
application server (called a "Composer" by Anoto) which turns the address 
field to ASCII and then creates an email with the contents of the "writing 
field" which is sent to the address field. The end result is that an email is 
sent to Mom@home.com. The pen system is flexible in that by using a 
different pattern on the paper, and directing the contents to a different 
Composer, a variety of different services could be created (for example a 
translation from English to Spainish prior to sending the email, or the 
addition of "Email", "FAX" or "SMS" check boxes to allow the message to 
be delivered through different means.). 



[0003] In the current implementations the paper is preprinted for a 
predefined application (like a notepad that provides the email functions 
described in the example above); it does not provide a mechanism to 
associate a printed piece of paper with an application in an ad-hoc fashion. 
For example, the current Anoto system can be used easily with pre-printed 
calendars, however a great majority of users print their calendars from a PC 
with their current itinerary. The current Anoto approach does not provide a 
good mechanism to allow the ad-hoc creation of printed material such that 
changes to this "ad-hoc" printed material (e.g., a calendar) are reflected back 
into the original electronic master. 



BRIEF DESCRIPTION OF THE DRAWINGS 
[0004] The present invention will be understood more fully from the 
detailed description given below and from the accompanying drawings of 
various embodiments of the invention, which, however, should not be taken 
to limit the invention to the specific embodiments, but are for explanation 
and understanding only. 

[0005] Figure 1 is a flow diagram of one embodiment of the process for 
automatically updating an electronic application when making edits to a 
hardcopy representation of that application. 

[0006] Figure 2 is a pictorial illustration of the process flow of printing a 
hardcopy representation of an electronic application. 

[0007] Figure 3 illustrates an example of a hardcopy representation. 

[0008] Figure 4 illustrates another example of a hardcopy representation. 

[0009] Figure 5 is a block diagram of one embodiment of a pen. 
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[0010] Figure 6 is a block diagram of one embodiment of a computer system. 



DETAILED DESCRIPTION OF THE PRESENT INVENTION 
[0011] A method and apparatus are disclosed for taking an electronic 
application program such as, for example, PowerPoint, Outlook, Windows, 
and Word of Microsoft Corporation of Redmond, Washington, or any PC 
application, and printing them on a piece of paper having a preprinted 
pattern thereon, thereby creating a hardcopy representation. The hardcopy 
representation of the application includes identification information (e.g., a 
unique ID). The identification information associates the application printed 
on the page with the preprinted pattern on the paper. Using a special pen, 
edits may be made to the hardcopy representation. The pen records these 
edits and sends the updates to a computer system automatically. In 
response to the receiving the edits, the computer system updates the 
electronic application automatically. 

[0012] In the following description, numerous details are set forth to provide 
a thorough understanding of the present invention. It will be apparent, 
however, to one skilled in the art, that the present invention may be 
practiced without these specific details. In other instances, well-known 
structures and devices are shown in block diagram form, rather than in 
detail, in order to avoid obscuring the present invention. 
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[0013] Some portions of the detailed descriptions that follow are presented 
in terms of algorithms and symbolic representations of operations on data 
bits within a computer memory. These algorithmic descriptions and 
representations are the means used by those skilled in the data processing 
arts to most effectively convey the substance of their work to others skilled 
in the art. An algorithm is here, and generally, conceived to be a self- 
consistent sequence of steps leading to a desired result. The steps are those 
requiring physical manipulations of physical quantities. Usually, though 
not necessarily, these quantities take the form of electrical or magnetic 
signals capable of being stored, transferred, combined, compared, and 
otherwise manipulated. It has proven convenient at times, principally for 
reasons of common usage, to refer to these signals as bits, values, elements, 
symbols, characters, terms, numbers, or the like. 

[0014] It should be borne in mind, however, that all of these and similar 
terms are to be associated with the appropriate physical quantities and are 
merely convenient labels applied to these quantities. Unless specifically 
stated otherwise as apparent from the following discussion, it is appreciated 
that throughout the description, discussions utilizing terms such as 
"processing" or "computing" or "calculating" or "determining" or 



"displaying" or the like, refer to the action and processes of a computer 
system, or similar electronic computing device, that manipulates and 
transforms data represented as physical (electronic) quantities within the 
computer system's registers and memories into other data similarly 
represented as physical quantities within the computer system memories or 
registers or other such information storage, transmission or display devices. 
[0015] The present invention also relates to apparatus for performing the 
operations herein. This apparatus may be specially constructed for the 
required purposes, or it may comprise a general purpose computer 
selectively activated or reconfigured by a computer program stored in the 
computer. Such a computer program may be stored in a computer readable 
storage medium, such as, but is not limited to, any type of disk including 
floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only 
memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, 
magnetic or optical cards, or any type of media suitable for storing electronic 
instructions, and each coupled to a computer system bus. 
[0016] The algorithms and displays presented herein are not inherently 
related to any particular computer or other apparatus. Various general 
purpose systems may be used with programs in accordance with the 
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teachings herein, or it may prove convenient to construct more specialized 
apparatus to perform the required method steps. The required structure for 
a variety of these systems will appear from the description below. In 
addition, the present invention is not described with reference to any 
particular programming language. It will be appreciated that a variety of 
programming languages may be used to implement the teachings of the 
invention as described herein. 

[0017] A machine-readable medium includes any mechanism for storing or 
transmitting information in a form readable by a machine (e.g., a computer). 
For example, a machine-readable medium includes read only memory 
("ROM"); random access memory ("RAM"); magnetic disk storage media; 
optical storage media; flash memory devices; electrical, optical, acoustical or 
other form of propagated signals (e.g., carrier waves, infrared signals, digital 
signals, etc.); etc. 

Overview 

[0018] The present invention provides a technique by which edits made to a 
hardcopy representation of an electronic application may be reflected 
automatically in the electronic application. Figure 1 is a flow diagram of one 
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embodiment of the process for automatically updating an electronic 
application when making edits to a hardcopy representation of that 
application. The process depicted in Figure 1 is performed by processing 
logic that that may comprise hardware (e.g., circuitry, logic, etc.), software 
(such as is run on a general purpose machine or a dedicated machine), or a 
combination of both. 

[0019] Referring to Figure 1, the process begins by processing logic printing 
a hardcopy representation of an electronic application on a paper that has a 
printed pattern and identification information thereon (processing block 
101). The printed pattern may be preprinted on the paper prior to the 
electronic application being printed on the paper or may be printed on the 
paper at the same time as the electronic application. In one embodiment, the 
printed pattern is a pattern of dots. The pattern of dots may be an Anoto 
pattern. Alternatively, the pattern may be applied to the paper using a 
technique other than printing. 

[0020] The identification information associates the printed pattern on the 
paper with the electronic application. In one embodiment, the identification 
information comprises an ID that is unique to the electronic application. In 



12 

one embodiment the ID is printed with an icon on the hardcopy 
representation. 

[0021] The identification information may be printed anywhere on the 
paper, such as, for example, the corner or side of the paper. In one 
embodiment, the identification information is printed in a corner of the 
hardcopy representation at a location that does not include any portion of 
the printed pattern. This is important to ensure that the pen is able to 
process the identification information without the potential confusion 
caused by the printed pattern. This allows the printed material to be 
associated with the pre-printed pattern on the paper by the user drawing a 
line in this ID field. The pen will recognize the ID pattern and the paper 
pattern, and will then be able to associate the paper (with its unique pattern) 
with the application that printed information on the paper (via the ID 
pattern). 

[0022] Using the hardcopy representation, processing logic records one or 
more edits made to the hardcopy representation using a pen by recording 
movements of the pen with respect to the printed pattern (processing block 
102). 
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[0023] In one embodiment, recording pen movements causes camera images 
of the printed pattern to be created. These camera images are used by the 
pen/ local PLS (a Paper Look-up server located on the PC) to calculate the 
exact position of the pen as the pen is used to create the one or more edits. 
Thus, the pen strokes themselves are not recorded, but portions of the 
printed pattern are recorded from which the edits are recreated by the 
processing in the pen. 

[0024] Thereafter, processing logic automatically sends any edits made to 
the hardcopy representation to the computer system from which the 
hardcopy representation of the electronic application was created 
(processing block 103). In one embodiment, the edits are sent to the 
computer system via wireless transmission. The wireless transmission that 
may be employed may comprise Bluetooth technology in its current form, 
but could be transmitted by almost any cable technology (serial cable, 
universal serial bus (USB), etc.). 

[0025] Once the edits have been received, an electronic document 
corresponding to the hardcopy representation is updated with the edits 
(processing block 104). 
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[0026] Prior to making edits, the user configures the hardcopy 
representation to the electronic application by performing a pen stroke over 
the identification information. That is, the user draws a line over or near the 
identification information (or over or near the icon that is printed with it, 
and this action causes the pen to record the identification information by 
noting the pattern of the paper and the pattern of the ID, thus establishing 
the relationship of associating the ID to the pre-printed paper pattern. 
[0027] In one embodiment, the printed pattern on the sheets is always the 
same regardless of the electronic application being printed thereon. 
However, that is not a requirement. Also, if the hardcopy representation 
comprises multiple pages, then there is a new ID put on each page. That is, 
if the hardcopy representation comprises a ten page document, then ten 
unique IDs are used and each ID represents the page on which it is printed. 
In such a case, the dots in the pattern have the same context, but the ID 
associates the unique context that was created on the computer system with 
the electronic application. 

[0028] In one embodiment, if a different printed pattern is used for each 
application, then an individual may make edits to multiple hardcopy 
representations of different electronic applications in succession without 
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having put a mark at or near the icons that are positioned near the 
identification information for each paper prior to making edits on that page. 
[0029] Figure 2 is an exemplary illustration of the process in which a 
computer 201 with an application 202 uses printer in conjunction with the 
paper 203 to print an application such as, for example, a calendaring 
program to generate hardcopy representation 205. Paper 203 includes the 
printed pattern (e.g., pattern of dots, Anoto pattern). 

[0030] Icon 206 is printed on the hardcopy representation. The unique ID is 
printed in the location on top of the icon in an area of the paper that does not 
include a preprinted pattern. 

[0031] In one embodiment the paper also includes control fields 207. Each of 
these control fields may be used to enable the pen to be used to select 
specific functions. These functions may include selection of colors, and 
control attributes (e.g. "Insert a comment", "Insert a footnote", ...). Figure 3 
illustrates one embodiment of a hardcopy representation 205. 
[0032] The hardcopy representation may also include fields that will be 
subjected to OCR. An example of such fields is shown as fields 401 in Figure 
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[0033] Figure 5 illustrates one embodiment of a pen that may be used to 
make edits to the hardcopy representation. Such a pen is similar to an 
Anoto pen. Referring to Figure 5, the pen includes an inkwell 501 for 
dispensing ink from the pen, a camera 502 to create images of the unique 
pattern (e.g., Anoto pattern) on the hardcopy representation as well as the 
ID, and a processor 503 coupled to the camera to control the operation of the 
camera in a manner well known in the art. Processor 503 is also coupled to 
memory 504 to store the images created by camera 502. A transceiver 505 is 
coupled to memory 504 to send captured images and other information to a 
computer system. In one embodiment, transceiver 503 comprises a 
Bluetooth transceiver that sends information stored in memory 504 to a 
computer system via wireless communication. A battery 506 is also shown 
which powers the components in the pen. 

[0034] When the pen draws a line across the ID icon, it reads the ID pattern 
and then the paper pattern, which are both part of this larger pattern 
discussed above. The ID icon pattern location will be associated with the 
printed application, while the paper pattern will be associated with a blank 
page function. When the pen recognizes a pen stroke between these two 
pattern areas, the local composer will then associate that paper pattern with 
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the meaning assigned it via the ID pattern. During the creation of the paper 
application (i.e., when it was printed), the computer will have stored paper 
pattern information associated with the functions to be performed (e.g., 
writing in this pattern area means to create an appointment). 
[0035] Figure 6 is a block diagram of an exemplary computer system that 
may perform one or more of the operations described herein. Referring to 
Figure 6, computer system 600 may comprise an exemplary client 650 or 
server 600 computer system. Computer system 600 comprises a 
communication mechanism or bus 611 for communicating information, and 
a processor 612 coupled with bus 611 for processing information. Processor 
612 includes a microprocessor, but is not limited to a microprocessor, such 
as, for example, Pentium™, PowerPC™, Alpha™, etc. 
[0036] System 600 further comprises a random access memory (RAM), or 
other dynamic storage device 604 (referred to as main memory) coupled to 
bus 611 for storing information and instructions to be executed by processor 
612. Main memory 604 also may be used for storing temporary variables or 
other intermediate information during execution of instructions by 
processor 612. 
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[0037] Computer system 600 also comprises a read only memory (ROM) 
and/or other static storage device 606 coupled to bus 611 for storing static 
information and instructions for processor 612, and a data storage device 
607, such as a magnetic disk or optical disk and its corresponding disk drive. 
Data storage device 607 is coupled to bus 611 for storing information and 
instructions. 

[0038] Computer system 600 may further be coupled to a display device 621, 
such as a cathode ray tube (CRT) or liquid crystal display (LCD), coupled to 
bus 611 for displaying information to a computer user. An alphanumeric 
input device 622, including alphanumeric and other keys, may also be 
coupled to bus 611 for communicating information and command selections 
to processor 612. An additional user input device is cursor control 623, such 
as a mouse, trackball, trackpad, stylus, or cursor direction keys, coupled to 
bus 611 for communicating direction information and command selections 
to processor 612, and for controlling cursor movement on display 621. 
[0039] Another device that may be coupled to bus 611 is hard copy device 
624, which may be used for printing instructions, data, or other information 
on a medium such as paper, film, or similar types of media. Furthermore, a 
sound recording and playback device, such as a speaker and/or microphone 
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may optionally be coupled to bus 611 for audio interfacing with computer 
system 600. Another device that may be coupled to bus 611 is a 
wired /wireless communication capability 625 to communication to a phone 
or handheld palm device. 

[0040] Note that any or all of the components of system 600 and associated 
hardware may be used in the present invention. However, it can be 
appreciated that other configurations of the computer system may include 
some or all of the devices. 

[0041] Whereas many alterations and modifications of the present invention 
will no doubt become apparent to a person of ordinary skill in the art after 
having read the foregoing description, it is to be understood that any 
particular embodiment shown and described by way of illustration is in no 
way intended to be considered limiting. Therefore, references to details of 
various embodiments are not intended to limit the scope of the claims which 
in themselves recite only those features regarded as essential to the 
invention. 



